Temporal difference (TD) learning is a simple algorithm for policy evaluation in reinforcement learning. The performance of TD learning is affected by high variance and it can be naturally enhanced with variance reduction techniques, such as the Stochastic Variance Reduced Gradient (SVRG) method. Recently, multiple works have sought to fuse TD learning with SVRG to obtain a policy evaluation method with a geometric rate of convergence. However, the resulting convergence rate is significantly weaker than what is achieved by SVRG in the setting of convex optimization. In this work we utilize a recent interpretation of TD-learning as the splitting of the gradient of an appropriately chosen function, thus simplifying the algorithm and fusing TD with SVRG. We prove a geometric convergence bound with predetermined learning rate of 1/8, that is identical to the convergence bound available for SVRG in the convex setting.

现实世界的顺序决策需要数据驱动的算法，这些算法在整个培训中为性能提供实际保证，同时还可以有效利用数据。无模型的深入强化学习代表了此类数据驱动决策的框架，但是现有算法通常只关注其中一个目标，同时牺牲了相对于另一个目标。政策算法确保整个培训的政策改进，但遭受了较高的样本复杂性，而政策算法则可以通过样本重用，但缺乏理论保证来有效利用数据。为了平衡这些竞争目标，我们开发了一系列广义政策改进算法，这些算法结合了政策改进的政策保证和理论支持的样本重用的效率。我们通过对DeepMind Control Suite的各种连续控制任务进行广泛的实验分析来证明这种新算法的好处。

We develop a simple framework to learn bio-inspired foraging policies using human data. We conduct an experiment where humans are virtually immersed in an open field foraging environment and are trained to collect the highest amount of rewards. A Markov Decision Process (MDP) framework is introduced to model the human decision dynamics. Then, Imitation Learning (IL) based on maximum likelihood estimation is used to train Neural Networks (NN) that map human decisions to observed states. The results show that passive imitation substantially underperforms humans. We further refine the human-inspired policies via Reinforcement Learning (RL) using the on-policy Proximal Policy Optimization (PPO) algorithm which shows better stability than other algorithms and can steadily improve the policies pretrained with IL. We show that the combination of IL and RL can match human results and that good performance strongly depends on combining the allocentric information with an egocentric representation of the environment.

抗微生物抗性（AMR）是患者的风险和医疗保健系统的负担。但是，AMR测定通常需要几天。本研究为基于易于使用的临床和微生物预测因子，包括患者人口统计，医院住宿数据，诊断，临床特征以及微生物/抗微生物特征，以及仅使用微生物/抗微生物特征将这些模型与微生物/抗微生物特性进行基于幼稚抗体模型的模型的预测模型。在培养之前准确地预测阻力的能力可以向临床决策提供通知临床决策并缩短行动时间。这里采用的机器学习算法显示出改进的分类性能（接收器操作特性曲线0.88-0.89的区域）与使用飞利浦EICU研究所的6个生物和10个抗生素的接收器操作特征曲线0.86下的接收器下的面积为0.88-0.89）（ERI ）数据库。该方法可以帮助指导抗菌治疗，目的是改善患者结果并减少不必要或无效抗生素的使用。

不受限制的在线线性优化（OLO）是研究机器学习模型培训的实用问题。现有作品提出了许多基于潜在的算法，但总的来说，这些潜在功能的设计在很大程度上取决于猜测。为了简化此工作流程，我们提出了一个框架，该框架通过求解部分微分方程（PDE）来生成新的潜在功能。具体来说，当损失是1-lipschitz时，我们的框架会产生一种新颖的算法，并随时随地遗憾绑定$ c \ sqrt {t}+|| || u || \ sqrt {2t} [\ sqrt {\ sqrt {\ log（1+|| |||/c）}+2] $，其中$ c $是用户指定的常数，$ u $是任何比较器未知和无限的先验者。这样的界限实现了最佳的损失重格折衷，而没有不切实际的tuble俩。此外，匹配的下限表明，包括常量乘数$ \ sqrt {2} $在内的领先订单项很紧。据我们所知，提出的算法是第一个实现此类最佳性的算法。

In recent years distributional reinforcement learning has produced many state of the art results. Increasingly sample efficient Distributional algorithms for the discrete action domain have been developed over time that vary primarily in the way they parameterize their approximations of value distributions, and how they quantify the differences between those distributions. In this work we transfer three of the most well-known and successful of those algorithms (QR-DQN, IQN and FQF) to the continuous action domain by extending two powerful actor-critic algorithms (TD3 and SAC) with distributional critics. We investigate whether the relative performance of the methods for the discrete action space translates to the continuous case. To that end we compare them empirically on the pybullet implementations of a set of continuous control tasks. Our results indicate qualitative invariance regarding the number and placement of distributional atoms in the deterministic, continuous action setting.

Data scarcity is one of the main issues with the end-to-end approach for Speech Translation, as compared to the cascaded one. Although most data resources for Speech Translation are originally document-level, they offer a sentence-level view, which can be directly used during training. But this sentence-level view is single and static, potentially limiting the utility of the data. Our proposed data augmentation method SegAugment challenges this idea and aims to increase data availability by providing multiple alternative sentence-level views of a dataset. Our method heavily relies on an Audio Segmentation system to re-segment the speech of each document, after which we obtain the target text with alignment methods. The Audio Segmentation system can be parameterized with different length constraints, thus giving us access to multiple and diverse sentence-level views for each document. Experiments in MuST-C show consistent gains across 8 language pairs, with an average increase of 2.2 BLEU points, and up to 4.7 BLEU for lower-resource scenarios in mTEDx. Additionally, we find that SegAugment is also applicable to purely sentence-level data, as in CoVoST, and that it enables Speech Translation models to completely close the gap between the gold and automatic segmentation at inference time.

The cyber-physical convergence is opening up new business opportunities for industrial operators. The need for deep integration of the cyber and the physical worlds establishes a rich business agenda towards consolidating new system and network engineering approaches. This revolution would not be possible without the rich and heterogeneous sources of data, as well as the ability of their intelligent exploitation, mainly due to the fact that data will serve as a fundamental resource to promote Industry 4.0. One of the most fruitful research and practice areas emerging from this data-rich, cyber-physical, smart factory environment is the data-driven process monitoring field, which applies machine learning methodologies to enable predictive maintenance applications. In this paper, we examine popular time series forecasting techniques as well as supervised machine learning algorithms in the applied context of Industry 4.0, by transforming and preprocessing the historical industrial dataset of a packing machine's operational state recordings (real data coming from the production line of a manufacturing plant from the food and beverage domain). In our methodology, we use only a single signal concerning the machine's operational status to make our predictions, without considering other operational variables or fault and warning signals, hence its characterization as ``agnostic''. In this respect, the results demonstrate that the adopted methods achieve a quite promising performance on three targeted use cases.

Automated Machine Learning-based systems' integration into a wide range of tasks has expanded as a result of their performance and speed. Although there are numerous advantages to employing ML-based systems, if they are not interpretable, they should not be used in critical, high-risk applications where human lives are at risk. To address this issue, researchers and businesses have been focusing on finding ways to improve the interpretability of complex ML systems, and several such methods have been developed. Indeed, there are so many developed techniques that it is difficult for practitioners to choose the best among them for their applications, even when using evaluation metrics. As a result, the demand for a selection tool, a meta-explanation technique based on a high-quality evaluation metric, is apparent. In this paper, we present a local meta-explanation technique which builds on top of the truthfulness metric, which is a faithfulness-based metric. We demonstrate the effectiveness of both the technique and the metric by concretely defining all the concepts and through experimentation.

In this paper, we address the problem of image splicing localization with a multi-stream network architecture that processes the raw RGB image in parallel with other handcrafted forensic signals. Unlike previous methods that either use only the RGB images or stack several signals in a channel-wise manner, we propose an encoder-decoder architecture that consists of multiple encoder streams. Each stream is fed with either the tampered image or handcrafted signals and processes them separately to capture relevant information from each one independently. Finally, the extracted features from the multiple streams are fused in the bottleneck of the architecture and propagated to the decoder network that generates the output localization map. We experiment with two handcrafted algorithms, i.e., DCT and Splicebuster. Our proposed approach is benchmarked on three public forensics datasets, demonstrating competitive performance against several competing methods and achieving state-of-the-art results, e.g., 0.898 AUC on CASIA.